Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: wcow: add support for bind and cache mounts #5708

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

feat: wcow: add support for bind and cache mounts #5708

wants to merge 1 commit into from
+201 −39

Conversation

profnandaa
Copy link
Collaborator

@profnandaa profnandaa commented Feb 6, 2025 •
edited
Loading

Currently, mounts are not supported for WCOW builds, see #5678. This commit introduces support for bind and cache mounts. The remaining two require a little more work and consultation with the platform teams for enlightment.

WIP Checklist:

  • Support for bind mounts
  • Support for cache mounts
  • add frontend/dockerfile integration tests
  • add client integration tests
  • add documentation
  • add note and plan on the missing feature(s) i.e. secret mounts (that needs tmpfs)
  • spec out / second attempt for SSH mount

Addresses part of #5678
Fixes #5603

Demo

Prep the context directory:

mkdir mounts-demo
cd mounts-demo
mkdir temp
echo "hello: root" > root.txt
echo "hello: inner" > .\temp\inner.txt

Dockerfile:

FROM mcr.microsoft.com/windows/nanoserver:ltsc2022 AS base
# this is needed for the access of the mounts
USER ContainerAdministrator

# cache mount example
RUN --mount=type=cache,target=/mycache echo "hello" > \mycache\foo
RUN --mount=type=cache,target=/mycache dir \mycache\foo

# bind mount examples
FROM base
# bind root of the context dir
RUN --mount=type=bind,target=/out type out\root.txt
# bind an inner directory (/temp) in the context
RUN --mount=type=bind,src=/temp,dst=/out type out\inner.txt

FROM wintools/nanoserver AS tools
RUN mkdir \bin
RUN copy \Windows\System32\whoami.exe \bin\whoami.exe


FROM base
# bind mount from another stage
RUN --mount=type=bind,from=tools,src=/bin,dst=/bin \bin\whoami.exe

Build command:

buildctl build --frontend dockerfile.v0 `
--local context=. --local dockerfile=. `
--output type=image,name=docker.io/profnandaa/mount-test,push=false `
--progress plain --no-cache

Build log:

#8 [base 2/3] RUN --mount=type=cache,target=/mycache echo "hello" > mycachefoo
#8 ...

#9 [tools 2/3] RUN mkdir bin
#9 DONE 2.8s

#8 [base 2/3] RUN --mount=type=cache,target=/mycache echo "hello" > mycachefoo
#8 DONE 2.8s

#10 [base 3/3] RUN --mount=type=cache,target=/mycache dir mycachefoo
#10 ...

#11 [tools 3/3] RUN copy WindowsSystem32whoami.exe binwhoami.exe
#11 8.133         1 file(s) copied.
#11 ...

#10 [base 3/3] RUN --mount=type=cache,target=/mycache dir mycachefoo
#10 8.217  Volume in drive C has no label.
#10 8.218  Volume Serial Number is 1E6D-2BB8
#10 8.218
#10 8.218  Directory of C:\mycache
#10 8.218
#10 8.222 02/06/2025  02:52 PM                10 foo
#10 8.222                1 File(s)             10 bytes
#10 8.222                0 Dir(s)  389,156,036,608 bytes free
#10 ...

#11 [tools 3/3] RUN copy WindowsSystem32whoami.exe binwhoami.exe
#11 DONE 8.6s

#10 [base 3/3] RUN --mount=type=cache,target=/mycache dir mycachefoo
#10 DONE 8.6s

#12 [stage-3 1/1] RUN --mount=type=bind,from=tools,src=/bin,dst=/bin binwhoami.exe
#12 4.449 user manager\containeradministrator
#12 DONE 4.9s

#13 exporting to image
#13 exporting layers

billywr
billywr reviewed Feb 6, 2025
View reviewed changes
Copy link

@billywr billywr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fix LGTM

@profnandaa profnandaa changed the title wip: wcow: hack complete for bind and cache mounts support feat: wcow: add support for bind and cache mounts Feb 7, 2025
This was referenced Feb 7, 2025
Open
Open
@danielnilsson9
Copy link

danielnilsson9 commented Feb 7, 2025 •
edited
Loading

I have done some testing on this with cache mounts and it works! But I'm a bit confused by the mount target path.
I'm running the binaries from this build: https://github.com/moby/buildkit/actions/runs/13193504481

It doesn't seem to be possible to specify an absolute path, specifying a full windows path "C:/Users/ContainerAdministrator/.conan2/p" does not work:

RUN --mount=type=cache,id=conan-cache-v1,sharing=locked,target=C:/Users/ContainerAdministrator/.conan2/p `
failed to create shim task: hcs::CreateComputeSystem si4wd93a8a3gf69wkaqhu86uz: The parameter is incorrect.: unknown

If I specify "/Users/ContainerAdministrator/.conan2/p" is seems to be relative to the current set WORKDIR and not the root of the C:/ disk which I think is inconsistent with the linux implementation.

@profnandaa
Copy link
Collaborator Author

profnandaa commented Feb 7, 2025 via email

@danielnilsson9
Copy link

Thanks :) Here is a small reproduce:

# escape=`

FROM mcr.microsoft.com/windows/servercore:ltsc2022

SHELL ["C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

USER ContainerAdministrator

WORKDIR /build

# Does not work, cache directory mounted in C:/build/cache instead of C:/cache
RUN --mount=type=cache,id=cache-mount-v3,target=/cache `
    Get-ChildItem -Path "C:\\" ; `
    Get-ChildItem -Path "C:\\build"

# Does not work, no directory mounted, not in C:/cache and not in C:/build/cache
RUN --mount=type=cache,id=cache-mount-v3,target=C:/cache `
    Get-ChildItem -Path "C:\\" ; `
    Get-ChildItem -Path "C:\\build"

Thanks for testing, let me take a look at this on Monday. --- // sent from a tiny device while on the move. forgive the tie pose.
…
On Fri, Feb 7, 2025, 15:18 Daniel Nilsson @.> wrote: I have done some testing on this with cache mounts and it works! But I'm a bit confused by the mount target path. I'm running the binaries from this build: https://github.com/moby/buildkit/actions/runs/13193504481 It doesn't seem to be possible to specify an absolute path, specifying a full windows path "C:/mycache" does not work: RUN --mount=type=cache,id=conan-cache-v1,sharing=locked,target=C:/Users/ContainerAdministrator/.conan2/p ` failed to create shim task: hcs::CreateComputeSystem si4wd93a8a3gf69wkaqhu86uz: The parameter is incorrect.: unknown If I specify "/Users/ContainerAdministrator/.conan2/p" is seems to be relative to the current set WORKDIR and not the root of the C:/ disk which I think is inconsistent with the linux implementation. — Reply to this email directly, view it on GitHub <#5708 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB7ZELBM76SSE6V7EIIZRD2OSQCFAVCNFSM6AAAAABWTGIERSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBSG43DQNBYGM . You are receiving this because you authored the thread.Message ID: @.>

@profnandaa
feat: wcow: add support for bind and cache mounts
261ee88 
Currently, mounts are not supported for WCOW builds,
see moby#5678. This commit introduces support for
bind and cache mounts. The remaining two require
a little more work and consultation with the platform
teams for enlightment.

WIP Checklist:

- [x] Support for bind mounts
- [x] Support for cache mounts
- [x] add frontend/dockerfile integration tests
- [ ] add client integration tests
- [ ] add documentatio
- [ ] add note and plan on the missing feature(s)
	i.e. secret mounts (that need tmpfs)
- [ ] spec out / second attempt for SSH mount

Fixes moby#5603
Addresses part of moby#5678

Signed-off-by: Anthony Nandaa <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@billywr billywr billywr left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
area/dockerfile area/executor area/frontend area/testing area/windows
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

WCOW: RUN with bind/cache mounts
3 participants
@profnandaa @danielnilsson9 @billywr